Existing cross-modal hashing algorithms underestimate the importance of semantic differences between different class labels and ignore the balance condition of hash vectors， which makes the learned hash codes less discriminative. In addition， some methods utilize the label information to construct similarity matrix and treat multi-label data as single label ones to perform modeling， which causes large semantic loss in multi-label cross-modal retrieval. To preserves accurate similarity relationship between heterogeneous data and the balance property of hash vectors， a novel supervised hashing algorithm， namely Discriminative Matrix Factorization Hashing （DMFH） was proposed. In this method， the Collective Matrix Factorization （CMF） of the kernelized features was used to obtain a shared latent subspace. The proportion of common labels between the data was also utilized to describe the similarity degree of the heterogeneous data. Besides， a balanced matrix was constructed by label balanced information to generate hash vectors with balance property and maximize the inter-class distances among different class labels. By comparing with seven advanced cross-modal hashing retrieval methods on two commonly used multi-label datasets， MIRFlickr and NUS-WIDE， DMFH achieves the best mean Average Precision （mAP） on both I2T （Image to Text） and T2I （Text to Image） tasks， and the mAPs of T2I are better， indicating that DMFH can utilize the multi-label semantic information in text modal more effectively. The validity of the constructed balanced matrix and similarity matrix is also analyzed， verifying that DMFH can maintain semantic information and similarity relations， and is effective in cross-modal hashing retrieval.